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ABSTRACT 

In March and May of 2002 two workshops on adolescent 
literacy, co-sponsored by several federal agencies and professional 
associations, were held. They included presentations, discussion panels, 
audience questions and comments, and small working groups. In the second 
workshop, examples of instructional models being implemented in middle and 
high schools were presented, highlighting the importance of the development, 
implementation, and evaluation of such models, and the value of rigorous 
design in evaluation. This paper summarizes the comments and issues that were 
raised about models of instruction for adolescent students. The paper 
discusses conditions of implementation; evaluation of models; instructional 
approach of models; and professional development for model implementation. 
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ADOLESCENT LITERACY: MODEL DEVELOPMENT, 
IMPLEMENTATION, AND EFFECTIVENESS 



3 In March and May 2002, two workshops on Adolescent Literacy were held. These workshops were co- 
il sponsored by several federal agencies and professional associations, and brought together 
I researchers, practitioners, administrators, funders and policy-makers. They included presentations, 
if discussion panels, audience questions and comments, and small working groups. In the second 
I! workshop, examples of instructional models being implemented in middle and high schools were 
I;; presented, highlighting the importance of the development, implementation and evaluation of such 
■! models, and the value of rigorous design in evaluation. Below is a summary of the comments and 
issues that were raised about models of instruction for adolescent students. 



Conditions of Implementation 



I 

I 



As literacy models must operate within the complex instructional and organizational structures of 
secondary schools, research is needed to identify the characteristics of middle and high schools 
where implementation is achieved and positively impacts student outcomes. There is also a need to 
discern the degree to which a range of literacy models can be adapted to accommodate school and 
student characteristics without diminishing the viability or effectiveness of the approach. Similarly, 
model developers and evaluators should measure the extent to which model implementation and 
effectiveness is impacted by inconsistencies in student and teacher behavior and participation. 




I 



• Some evidence for and against the benefits or costs of implementing the model is needed: 
when the model is implemented, does it detract from instruction in other areas? Cost- 
effectiveness relates to the severity and extent of the literacy problem. 

• What theory of knowledge transfer shapes the model? 

• Are the models dynamic or static? Can the schools mold it to their needs and their students? 
Are these product or process approaches? 

• How closely linked are the conditions of implehnentation and success of implementation? 
Consider the complications of student attendance, absenteeism, turnover, and transience; 
teacher consistency; and English-language learners. 



Evaluation of Models 



Evaluations of models should represent partnerships among practitioners, model developers and 
researchers. Models should be developed to address literacy instruction for English language 
learners. Research is needed to extract key principles across practice models that can be used to 
create a menu of strategies for assisting students at different levels of ability. There is also a need 
for evaluative data on models that appear to interrupt the plateau in achievement often reached by 
disabled students. 
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• Models should be compared to models instead of to no treatment groups. This requires some 
consensus definitions and specific criteria for seiecting comparison groups for evaluations of 
interventions. Aiso, the theoretical base of the instructional approach in the model must be 
clearly expressed in order to do the comparisons. 

• Conducting evaluations in the setting and reality of a school system is chailenging. Effective 
management of student populations/subjects will be needed to ascertain the effectiveness of 
the interventions. This wili bear on the deveiopment of longitudinai studies, ciinical triais, and 
carefui seiection of outcome measures. Studies must be designed with ciear attention to 
issues of sampie size and power, and the unit of analysis (e.g., teacher, ciass, schooi, 
students). The importance of designing solid effectiveness triais depends directiy on how 
extensiveiy the intervention is being implemented. 

• To know if practicai and permanent change has occurred, long-term follow-up data are 
needed, and outcomes should be assessed beyond in-school academic achievement. What is 
the student's performance a year later? Has there been slippage? Do the effects continue and 
what conditions are necessary for that to continue? What conditions are needed (necessary) 
for sustained gains over time? 

• Model implementation and outcomes measured must be linked, cieariy described, and 
reported. In order to ascertain the fidelity of impiementation, changes in daiiy practice must 
be measured: what has really happened? Does the actual practice change on a day-to-day 
basis? Does the model disappear but leave a changed (improved) practice? Some toois for 
recording impiementation are teacher surveys, checklists, observation protocois, ciassroom 
observations, anecdotai records (student and teacher). School-ievei measures shouid inciude 
dropout rates and absenteeism. 

• Ciarity is needed about the subject population (e.g., how students are seiected, assessed, and 
piaced) for any modei. This wiii be important in attempting to generalize the use of the model 
to other groups of students. 

• What is the role of standardized tests in modei evaiuation? Is there a baseiine assessment 
that ascertains what students will be able to do? Sometimes students are not incapable, just 
under-practiced or lacking instruction. How is that assessed? 

Instructional Approach of Models 

Although there is a robust knowiedgebase on the development of reading skiiis during eariy 
childhood and the core components of instruction supporting the development of reading proficiency 
in grades K-3, there is littie evidence to suggest whether or not these research findings predict the 
efficacy of middle and high school literacy models. In constructing new iiteracy programs for 
adolescents and scaling up existing modeis of practice, model developers should clearly define the 
research-base for and pedagogicai approach of these iiteracy frameworks and identify core 
instructionai strategies. In doing so, model developers should seek to distinguish those practices that 
have optimai impact on the reading proficiency of students of different ability ieveis from those with 
minimal or no impact on student outcomes. The student population- as characterized by ability-ievel, 
learning differences, and Engiish language proficiency- shouid be defined at the outset of model 
development, in addition to the mode of deiivery (e.g. acceierated reading intervention; ninth-grade 
literacy course; strategies for reading in the content areas.) 

• Interventions should build on the existing research base. First, the research and theoretical 
bases must be clearly described or assessed. How generalizable and applicable to adolescent 
literacy is the reading research that aiready exists? How generaiizable are the five major 
components of early reading? Do they apply to adolescence? Examination of extant data 
would inform the definition of the problem and hypothesis generation for inquiry into effective 
interventions. Data sources couid include: NAEP, NALS, and military entrance exams. 

• What are the core instructionai strategies in the modei? How do they reiate to content and 



instruction across the curriculum? Some of the instructional components common to the 
practice models are self-selected, reading, teacher modeling, vocabulary development, and 
cooperative learning. Is there any consensus on which of these components are central and 
core? The definition of the problem Is still anecdotal thus compounding the difficulty of 
knowing how much and when and what kind of instruction is needed in each of the 
components of reading in order to develop a proficient reader. For example, how much should 
be isolated skill instruction vs just practicing reading? 

• The model must respond to the level of severity of literacy difficulty. For whom is the specific 
intervention or model being developed-for special needs students or for all students but 
developmentally appropriate to subgroups? 

Professional Development for Model Implementation 

The effectiveness of implementation may depend on the quality of a model's professional 
development program as measured by its impact on teacher behavior, engagement and consistency 
in delivery. Professional development programs should be founded in a solid research and theoretical 
base on adult learning. Model developers should define the recipients of the professional 
development (e.g. reading specialists; content area teachers; administrators) and identify the most 
effective modes of training for each group (e.g., in-class coaching; university courses; on-going 
seminars and teacher working groups.) Model developers should consider the needs of both pre- and 
in-service teachers and school administrators. 

• Flow much time is required for professional development? 

• Is it important to demonstrate the impact of professional development on the effectiveness of 
implementation? If so, this information would contribute to school-based decisions on how 
much time and how many resources are to be allotted to the program. 

• Is the delivery of professional development static or dynamic--meaning, how does it 
contribute, or not, to the ongoing improvement of implementation? 

• Does the model allow teachers to use what they already know, to be informed decision 
makers? What decisions can teachers feel free to make that will not alter the effectiveness of 
the model as tested? If the intervention is complicated, teachers may be reluctant to use it; 
thus work with teachers to obtain their "buy-in" may be crucially important. Flow does 
scripting interact with the teacher's need to be an informed decision-maker? 

• Research on persuasion, motivation, conceptual change, and engagement can be used to 
inform instructional practices for teachers. 
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